the kth visit in semi-markov processes

The KTH Visit in Semi-Markov Processes

Semi-markov Decision Processes

Considered are infinite horizon semi-Markov decision processes (SMDPs) with finite state and action spaces. Total expected discounted reward and long-run average expected reward optimality criteria are reviewed. Solution methodology for each criterion is given, constraints and variance sensitivity are also discussed.

Quantum semi-Markov processes.

We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes. General conditions for the complete positivity of the corresponding quantum dynamical maps are formulated. The resulting non-Markovian quantum processes allow t...

Semi-Markov Decision Processes

The previous chapter dealt with the discrete-time Markov decision model. In this model, decisions can be made only at fixed epochs t = 0, 1, . . . . However, in many stochastic control problems the times between the decision epochs are not constant but random. A possible tool for analysing such problems is the semiMarkov decision model. In Section 7.1 we discuss the basic elements of this model...

Some Peculiar Semi-markov Processes

Interactive Generalized Semi-Markov Processes

We have previously introduced Generalized Semi-Markovian Process Algebra (GSMPA), a process algebra based on ST semantics which is capable of expressing durational actions, where durations are expressed by general probability distributions. In this paper we introduce the calculus of Interactive Generalized Semi-Markov Processes (IGSMP), a variant of GSMPA where synchronizable actions are restri...